feat(insertion-normalization): un-merge CC's join-moves, and stop a re-served entry from re-binding - #295
Draft
Gunther-Schulz wants to merge 51 commits into
Draft
Conversation
…story rewrites stop busting the cache Claude Code re-serializes <system-reminder> hook blocks inside otherwise-stable user messages later in the session — moving one into its own message or merging it into a neighbour — which edits history mid-prefix and re-bills everything after the edit (reported independently as anthropics/claude-code#76606; measured here as the splice/insert-mid class, ~40 kB re-billed per unmitigated hit on a real session). The extension keeps a per-conversation canonical model of the message history keyed by content identity (message-hash.mjs: content hash + occurrence ordinal — position-independent, so repeated identical reminders stay distinct). Incoming volatile blocks are pinned to their first-seen serialization: when CC re-shapes an old reminder, the forwarded bytes keep the canonical form and the prefix survives. A history that stops matching the model (compaction, true rewrites) resets honestly rather than forcing a stale canon — pins survive the reset, order assumptions do not. Gated off by default: CACHE_FIX_INSERTION_NORMALIZE=1 enables normalization, CACHE_FIX_VOLATILE_PIN=1 the pinning. State persists under the state dir and survives proxy restarts. Measured on live traffic (513-request session, 2026-07-28): every observed splice/insert-mid pair forwarded with 0 re-billed bytes; the canonical-order invariant, cross-request stability and sequence gates all report 0 violations over 2.5 GB of captures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lete the pin (anthropics/claude-code#76606) When CC migrates a hook reminder out of its tool_result into a standalone system message mid-history, the pin restores the first-seen inline form — but the migrated copy still forwarded, splicing the same content in twice (measured live: ~61 kB splice, 124k tokens re-billed on one turn). Now a standalone message whose wrapper-normalized bytes equal a live pinned block is suppressed: never forwarded, never given a canonical identity. Genuine changes (normalized bytes differ) still forward and reset per the existing rule; assistant-role messages are excluded on principle. Suppression is re-detected each request from the pin set — no new state file. One event line per suppression rides the insertion event log. The real-pair red-green check in the new test file needs the replay tooling and capture; in this slice it skips, and runs where the tools land (cnighswonger#276). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
…ture (slice of fork 2dfe0f0) Path-scoped slice of fork commit 2dfe0f0: only the insertion-suppression real-pair test and the pinned fixture it falls back to. The same commit's harvest-pin.test.mjs and mitigation-output-form.test.mjs changes belong to the verification-tools slice (cnighswonger#276) and are not part of this PR. Co-Authored-By: Claude opus-5 <noreply@anthropic.com>
…pinned blocks — the 587k's shape CC sometimes migrates ALL of a message's volatile blocks out together, joined into one standalone message (both hook reminders, wrapper-stripped, joined with "\n\n"), rather than one standalone per block. The existing single-block suppression set could never match that shape. Each pinned entry with >=2 volatile blocks now also registers a join-hash — its blocks' unwrapped texts, in wire order, joined with the one observed separator — and findSuppressibleDuplicate checks it as a second pass. No subset-merges, no speculative separators: only the one shape measured live (capture s-633915a8, msg863/864, and independently confirmed on a second real occurrence at msg640/641 the same session). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit 78940a0)
…cate is payload (insertion slice of fork e0f8fcb) Path-scoped slice of fork commit e0f8fcb: the insertion-normalization tail guard and its tests. The same commit's output-guard assistant-terminal invariant (proxy/extensions/output-guard.mjs, test/output-guard.test.mjs) belongs to the output-guard slice (cnighswonger#278) and is not part of this PR. Co-Authored-By: Claude opus-5 <noreply@anthropic.com>
… fork da9bf8c Makes the file slice-portable: without tools/ the real-pair check now reaches its designed skip instead of dying at module load. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcivCe2iLnKZxpB4qTXzEb
…test reads (fork b1f7c58) The merge-suppression test does a top-level read of this harvested, sanitized fixture (16KB, no addresses); without it the file dies at load. The fork-only exclusion list names only LEDGER-*.json — this fixture is public on the fork and rides with its test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcivCe2iLnKZxpB4qTXzEb
…story rewrites stop busting the cache Claude Code re-serializes <system-reminder> hook blocks inside otherwise-stable user messages later in the session — moving one into its own message or merging it into a neighbour — which edits history mid-prefix and re-bills everything after the edit (reported independently as anthropics/claude-code#76606; measured here as the splice/insert-mid class, ~40 kB re-billed per unmitigated hit on a real session). The extension keeps a per-conversation canonical model of the message history keyed by content identity (message-hash.mjs: content hash + occurrence ordinal — position-independent, so repeated identical reminders stay distinct). Incoming volatile blocks are pinned to their first-seen serialization: when CC re-shapes an old reminder, the forwarded bytes keep the canonical form and the prefix survives. A history that stops matching the model (compaction, true rewrites) resets honestly rather than forcing a stale canon — pins survive the reset, order assumptions do not. Gated off by default: CACHE_FIX_INSERTION_NORMALIZE=1 enables normalization, CACHE_FIX_VOLATILE_PIN=1 the pinning. State persists under the state dir and survives proxy restarts. Measured on live traffic (513-request session, 2026-07-28): every observed splice/insert-mid pair forwarded with 0 re-billed bytes; the canonical-order invariant, cross-request stability and sequence gates all report 0 violations over 2.5 GB of captures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ions via the mid-conversation beta Every ToolSearch/deferred-tool load makes Claude Code re-send a different tools[] array. tools[] heads the cache prefix, so each load re-bills the entire context — the class reported in anthropics/claude-code#81967 (there triggered by LSP add/remove; the deferred-tool path hits it far more often). Anthropic's API already has the fix — the documented mid-conversation-tool-changes-2026-07-01 beta — and Claude Code 2.1.220 ships that beta's documentation in its own binary without using it on the wire. The extension freezes tools[] at its first-seen form per session (keyed session + system-prompt + conversation, so subagents and sidecars never share a baseline). A mid-session addition keeps the frozen bytes and instead announces the new tool with a tool_addition system message anchored at the tail, re-injected at a stable position on every subsequent request, with the beta header added. Removals and reorders are held byte-stable outright; a schema change resets honestly. Announcements are opt-in per MODEL, with evidence required: the beta is rolled out per model family and an unsupported one rejects the whole request with a 400 (measured on claude-sonnet-5 and claude-haiku-4-5; the haiku error names the gating capability — mid-conversation system content). An unknown model degrades to forwarding the changed tools[] — the status-quo bust, never a lost request — and the first suppressed announcement per model warns with the way out. tools/probe-tool-addition.mjs measures a candidate model in one real request; CACHE_FIX_TOOL_ADDITION_EXTRA admits a candidate on a throwaway proxy for the live probe. Allowlisted with wire evidence: claude-opus-5, claude-fable-5. Gated off by default: CACHE_FIX_TOOL_REWRITE=1. Stacked on pr/insertion-normalization (message-hash identity + system-prompt sub-key). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cheduled sweep Machinery for anyone running this proxy to verify it against their OWN traffic instead of trusting ours: replay re-runs the real extension pipeline offline over a capture, per-extension mutation attribution by hashing between stages, four cross-request invariants (stability, safety, sequence, canonical order), fidelity against the recorded forwarded-body hashes (five populations, never one ratio — 0/0 must not read as "checked and clean"), and --census classification of every consecutive same-conversation pair, with mitigation gaps priced in re-billed bytes rather than counted. harvest promotes structurally novel capture pairs into sanitized, committable fixtures (novelty judged against per-machine ledgers, so N machines contribute without duplicating classes). gate-live runs the replay gate over live captures on a schedule, under the SERVING gate set read from the running unit — never extension defaults — with heap- capped children (the cap is a memory-regression check, proven red) and a status file a checker can consume. cache-sim prices post-pipeline bytes against raw. read-lines is the shared pull-based line reader all of them use: readline's async iterator buffers the whole remaining file once the consumer awaits (measured 3.27 GB peak on a 1.5 GB capture under code that called itself streaming); the bite test pins bytesRead against consumed bytes and went red on line 3 against the readline shape. docs/dev-loop.md records the working discipline the tools enforce. Fixtures are sanitized (harvest strips content to structure) and were audited before publishing. Stacked on pr/deferred-tool-rewrite; pairs with pr/request-capture, which produces the input format. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hape watch, hardened sweep The branch was cut before an afternoon in which the daily sweep and its own findings changed these tools materially; as opened, the PR shipped a safety checker with a known bug (the declared-injection exemption filtered the output side only — an input carrying an injection-shaped message, e.g. a chained proxy feeding the pipeline its own output, read as a message drop nobody made; caught by the first stamped sweep and fixed with a both-ways bite). What this refresh brings, each with its tests: - safetyViolation filters declared injections on BOTH sides; - findSuccessions: conversation-boundary classification (compaction / resume-shaped / fork) with opener pricing; interleaves structurally suppressed (never-returns + first-appearance conditions, both bite-proven — the one-shot-sidecar phantom was caught by its own test); - census edit rows carry lastHumanAt/anchorDelta (the relation that attributed the mid-history edit population to reminder anchoring), with far-from-anchor rows flagged and their bytes excerpted to local stdout only; - harvest: shape watch counters (dormant thinking classes, baseline prefix sizes) and growth-step snapshots (evidence frozen, scrubbed, before capture rotation eats it); shape-verdicts.mjs is the consumer so the new ledger fields are never orphan telemetry; - gate-live: --census on every sweep, rows that compared zero pairs marked proves-nothing (ok requires a proving row), and the status file stamps the proxy/tools source fingerprints it exercised — a verdict that names its config but not its code stays "fresh" across code changes it never saw. proxy/source-fingerprint.mjs is included for the stamp; identical file ships in the capture PR — merges cleanly in either order, same precedent as the shared fixtures. docs: dev-loop's closing-gate section and the consumer setup page ride along. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, output-form metric, suppression exemptions The gate's mitigation metric was input-side only: it trusted the extension's self-report and never compared what was actually forwarded. Now each mitigation row carries outputForm (append / splice@N / edit@N), outputPreserved, and rebilledOutBytes — measured on the forwarded bytes — which is how a "mitigated" pair that still re-billed 124k was caught. The census classifies reminder block-migrations (inline <-> standalone) on splice and edit rows, and the safety and stability checks gain telemetry-sourced exemptions for the new suppression (a removed message has no shape to detect after the fact, so exemption keys off the extension's own suppression records). Extension synced to the cnighswonger#272 tip; real-pair red-green tests run in this slice, where tooling, extensions, and capture meet. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
…tput metric; extensions synced Three replay improvements from operating the gate: an unmissable stderr warning when a gated capture replays under default gates (the instrument error that booked a wrong verdict three times in one day — and whose first live fire caught the operator's own gateless replay); a --gates-from-capture flag applying the all-boot-records union so nobody hand-extracts gates; and outputForm now strips cache_control before comparing (a moved cache marker is not a content splice — five pairs totalling ~0.6 MB of phantom "re-billed splice" were CC's own benign marker relocation). Extensions synced to the cnighswonger#272/cnighswonger#273 tips so the slice's real-capture tests exercise the actual pipeline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
Real-pair tests (test/insertion-suppression.test.mjs, test/mitigation-output-form.test.mjs) replay a specific live capture and SKIP once it rotates out of the retention window. `node tools/harvest.mjs --pin <key> <n..m>` freezes the sanitized range as a committed, rotation-immune fixture at test/fixtures/harvested/pinned-<key>-<n>-<m>.json, reusing the existing scrubRecord sanitizer (never a second scrubber). Both real-pair tests replay their capture from request 0, not from n, because insertion-normalization keeps per-conversation canonical state that only reconstructs correctly if every prior request replayed in order. So the fixture holds every record (boot, outcome, request) from the start of the file through request m inclusive, not just n..m -- n..m names the pair under test, not a truncation point. Stated explicitly in the fixture's header. Adds parsePinRange/pinRange/runPin/readPinnedFixture and the --pin CLI flag; test/harvest-pin.test.mjs covers the mechanism on a tiny synthetic capture (range parsing, sanitization, CLI end-to-end, and the readPinnedFixture reader's [n, line] tuple parity with readCapture). Fallback wiring in the two real-pair tests lands in a follow-up commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit da4e7e1)
…nstant Deviation found while validating the first real --pin (capture s-633915a8, n=26->28): scrubText collapsed every <system-reminder>-wrapped block to the literal constant "REDACTED", regardless of its real content. That breaks insertion-normalization's wrap/unwrap cross-identity check (findSuppressibleDuplicate/unwrapVolatileText, proxy/extensions/ insertion-normalization.mjs) -- CC sometimes migrates a reminder OUT of its wrapper into a standalone duplicate message, and suppression fires only when the wrapped original's stripped bytes hash-match the standalone duplicate's bytes. Under the fixed constant, the wrapped original always hashed to "REDACTED" while the unwrapped duplicate hashed its real text independently, so the two never matched post-scrub. Measured directly: replaying the sanitized n=26->28 fixture through the real pipeline gave suppressed=0 / outputForm="splice@31" (the pre-fix defect shape) instead of the live capture's suppressed=1 / outputForm="append". The committed pinned fixture would have silently reproduced stale, wrong behaviour once the live capture rotated away -- worse than the SKIP it was meant to replace. Fix: a wrapped reminder now re-wraps scrubText's own recursive token for its inner text instead of a fixed placeholder. The wrapper tags still survive verbatim (any check that only tests for wrapper PRESENCE is unaffected -- verified against test/harvest.test.mjs's existing "wrappers survive" test, unchanged and still green), and two reminders with equal real bytes -- wrapped or not -- now hash equal after scrubbing, matching what the sanitizer already guarantees for ordinary text. Verified: test/harvest.test.mjs (14/14, no regression), and the fixed scrubber's output re-run through the actual pipeline (findMitigationGaps, findSafetyViolations) reproduces the live capture's real values exactly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit fda83cc)
…eating the evidence Wires the fixture-fallback into both real-pair tests (test/insertion-suppression.test.mjs, test/mitigation-output-form.test.mjs): live capture present -> unchanged path (readCapture); capture absent -> fall back to the pinned fixture via readPinnedFixture if one exists; both absent -> skip with a stated reason, as before. Both readers yield the same [n, line] tuple shape, so the replay loop in each test is unchanged either way. Both the capture path and the fixture path are overridable via CACHE_FIX_TEST_CAPTURE_OVERRIDE / CACHE_FIX_TEST_FIXTURE_OVERRIDE so the fallback's own red-green test never has to touch the real capture file, which is read-only evidence shared with other concurrent work. Commits the first real pin: test/fixtures/harvested/pinned-s-633915a8-26-28 .json, the n=26->28 pair both real-pair tests already reference in their comments, produced with `node tools/harvest.mjs --pin s-633915a8 26..28` (holds the full prefix 0..28 per the previous commit's replay-from-start finding) and verified against the fixed scrubber from the prior commit: replaying it through the actual pipeline reproduces the live capture's exact values (mitigated=true, outputForm="append", suppressed=1 at index 31, 0 safety violations). test/harvest-pin.test.mjs gains the fallback's red-green proof: both real-pair test files are run as actual subprocesses (not re-derived) with env overrides -- capture+fixture both absent skips (never a false pass), capture absent + the real committed fixture present runs and PASSES (never a false fail). Both real-pair suites are green in both modes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit 2dfe0f0)
… + tail guard Brings proxy/extensions/insertion-normalization.mjs and its real-pair test to the cnighswonger#272 slice tip (fork e0f8fcb): the merged-standalone join-hash set and the tail-position suppression guard. The extension rides here so this slice's tools replay the same behaviour cnighswonger#272 ships. Co-Authored-By: Claude opus-5 <noreply@anthropic.com>
…r path (slice of fork 2a11487) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcivCe2iLnKZxpB4qTXzEb
…nswers daily findDuplicateRequests classifies every ADJACENT same-conversation pair whose incoming message array is byte-identical (same length, same per-message hash at every index) — the "hidden duplicate request" falsifier (CC#78420) that was previously a throwaway python scan, now a census kind that rides every --census run (BACKLOG "Duplicate-request probe -> census check (Q1)"). Rides both JSON and text output alongside the existing toolsDeltas/edits/blockMigrations census kinds. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit 7bc3462)
…, log on staleness Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit 083c5d6)
… set Out-of-band runs (operator shell, doctor) read gates from an env that never carries them, so every gate-dependent verdict warned "off" on a serving machine. Env-set still wins; unset falls back to cache-fix-gate-status.json's recorded gates. Includes the import the first draft's own catch was masking (a ReferenceError swallowed into "no status file" — the fail-open hid the failure). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH (cherry picked from commit 8e91c69)
…ts on telemetry, never on shape fresh-session-sort's relocate branch prepends content to messages[0] on a block type's first appearance in the array — a deliberate one-time bust (the #34629 class) that reads to replay's cross-request stability check as a self-inflicted byte flip (n=2024->2025 in s-58c979ce: CC diverges at index 1, our output diverges earlier, at index 0). The extension now reports what it did (ctx.meta.freshSessionSortStats: relocated block types + a firstAppearance flag computed from the SAME backward scan that already finds each type's latest instance, plus the target message index). replay's stability check gains a telemetry-keyed exemption mirroring suppressedIndices' discipline: exempt only when the CURRENT entry's telemetry names the exact outDiv AND at least one relocated block is a genuine first appearance — never re-derived from divergence shape alone, so a relocation reported without telemetry, or reported as a recurrence, stays a violation. Exempted entries are reported separately (findStabilityExemptions, --json output, console section) rather than silently dropped. Verified: unit bites both ways (with telemetry -> exempt; without, or with firstAppearance:false -> still a violation), red-before-green on both the extension telemetry and the checker exemption. Real-pair replay on s-58c979ce confirms PRE: exactly 1 stability violation at n=2024->2025, attributed fresh-session-sort (measured independently twice). POST: 0 violations, 1 exemption annotated, safety/sequence/order violations unchanged at 0 (confirmed by the dispatcher's run on this working tree). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit e41e068)
…hared subset forwardedStable compares the whole forwarded tools[] array, so a genuine new-tool announcement always reads as "unstable" even when every tool CC already knew about round-tripped byte-identical. heldStable narrows the claim to the SHARED-name subset of a pair's forwarded tools (the tools present on both sides), the guarantee deferred-tool-rewrite actually makes. gate-live's toolsDeltas summary carries it beside forwardedStable. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit 813edc8)
…p — born with its reader Also pins the gate's real enable value in place: "on", not "1" — the booked =1 flip would have silently left the extension off. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH (cherry picked from commit 9876fff)
…oded count bit its author The sixth-row commit shipped red because the suite pipe reported grep's exit, not the test's; the two enumerations (count 8, five-name list) were exactly the corpus's hardcoded-count anti-pattern, now table-derived so a legitimate row cannot redden them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH (cherry picked from commit 7790dff)
… fork da9bf8c Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcivCe2iLnKZxpB4qTXzEb
…e refuted
The BACKLOG item this fixture was harvested for ("per-block standalone
match") assumed the flap's two standalone legs were two per-block copies
of one pinned reminder, which pinnedBlockHashes could not match. Measured
against the real bytes (capture s-0d6f38ba, n=102/104/105/108), all three
parts of that premise are false:
- pinnedBlockHashes ALREADY registers each volatile block's
individually-unwrapped text hash — it has since the original #76606
suppression, before the join-hash sibling (78940a0);
- the standalone legs are not per-block copies. msg86 is the JOIN of
msg85's four reminder blocks ("\n\n", 1256 chars) — which the
join-hash already matches — and msg94 is msg92's single reminder
(683 chars) — which the per-block hash already matches. Executed:
findSuppressibleDuplicate returns a hash for BOTH;
- the leg that matches nothing is msg91: a CROSS-MESSAGE join, msg89's
unwrapped reminder + "\n\n" + the whole standalone system message
that followed it at msg90 (1106 chars). No hash set spans two source
messages, and 78940a0's guard (c) refuses cross-entry joins by
design.
So suppression is not where the flap escapes. classifyPinned returns
reset("edit-shaped") at the isEdit co-location test BEFORE the
suppression pass runs — msg91 lands in the gap left by dropped msg90,
which is exactly the shape that test calls a genuine edit. Confirmed by
the shipped pipeline itself, not only offline: replay under the
capture's own boot gates (10/10) logs n=104 reason=edit-shaped, and the
census still lists all three flap pairs.
The fix therefore needs a design decision above a build brief — whether
a cross-message join may suppress at all (suppressing msg91 would drop
msg90's bytes from the wire entirely, since its canonical entry is
dropped and this extension never re-adds a message) and whether
suppression may run before the edit-shaped reset. No code change here.
The fixture is the evidence, taken before the capture rotates: full
message arrays for all four requests of the three flap pairs, so both
the suppression relations and the reset decision reproduce offline.
Sanitized via harvest's scrubMessage, EXCEPT the hook-reminder texts
that participate in the migration, kept raw — scrubText is not a
homomorphism over concatenation, so a merged standalone would scrub to
a token unrelated to its parts and the join relation, the point of the
fixture, would not survive. Every raw text was read in full first: all
are harness-generated hook reminders, no operator prose and no
host/network identifiers. Fidelity checked against the raw bytes: same
action, same reset reason, same per-message suppression verdicts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 090a110)
…quests is flagged, not read off adjacent lines The 2026-07-30 221k event (threat matrix row 4, session 0d6f38ba, n=102->104->105->108 in 11 seconds) was visible only by reading three adjacent census lines and noticing the direction column alternate. A one-way migration is absorbable by the volatile pin; an oscillation is not, when the pin classifies only one of the two shapes — it busts on every second flip at best. That distinction now has a name in the census instead of living in a reader's attention. blockMigration rows carry the migrating block's `hash` (the unit hash scanBlockMigrations already computed — no second notion of sameness), and the flap scan runs per conversation group, so the window counts requests of that CONVERSATION, not of the wire: cache prefixes are per-conversation, and a co-tenant's traffic is not part of the clock. Red-first, three mutations each biting its own test: neutralised markFlaps -> the two BITEs and the boundary case go red; window bound removed -> the 6-requests-later guard goes red; block-hash identity dropped -> the different-blocks guard goes red. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit fc44da3)
…e that shed a sibling is not a standalone emergence Measured on the real flap bytes (fixture flap-s-0d6f38ba-86.json, pair n=102->104): PREV[92] user [tool_result, text(<system-reminder> 720 chars)] CUR [93] user [tool_result] <- PREV[92] having SHED its reminder CUR [94] system "…" (683 chars) <- PREV[92]'s reminder, unwrapped Both guards of the existing definition were satisfied by a block that never moved. Two messages were inserted above, so the host's own index moved and the same-position guard compared against an unrelated message; and `standalone` is `blocks.length === 1`, which a message that SHRANK to one block satisfies. The tool_result was therefore reported as migrating 92->93 while sitting exactly where it always was. The census reported 6 migrations on that capture where 3 exist, and after the flap annotation each phantom also carried a `flap` tag — telling the reader that two blocks oscillate when one does. Candidacy condition restored from the class the section already names (the reminder swap): a block is a migration candidate only where it appears <system-reminder>-WRAPPED on its INLINE side — as the source unit when leaving a multi-block message, as the destination unit when joining one. This narrows the check to its own declared subject rather than adding a new rule. Note this is NOT the shrink predicate inside blockUnits: whether a message shrank is not knowable from the message alone, and pair-locally it is not knowable at all for the reverse direction — in pair n=104->105 the one-block msg93 is simply a one-block message, so a shrink-based rule leaves the standalone->inline phantom standing. The wrapper is pair-local and definitional. Red-first, and probed for over-narrowing rather than only for the fix: all three bites go red with the condition removed — the fixture bite with "6 !== 3" on the real bytes — and green with it; the documented-real n=26->28 migration (30->31) survives untouched on s-633915a8, as does the second flap there (105->107->108->109), and only one further phantom drops (n=1515->1528). Gate 0/0/0/0 on both captures under their own boot gates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 47defba)
…, or red The four existing gates are all positional: did our bytes move earlier than CC's, did we change the message sequence, did a normalize get followed by a reset, does canonical order track the wire. None of them can see a message CC sent that we never forwarded and whose content exists nowhere else, because a deletion leaving the survivors positionally consistent is invisible to all four. pin-and-suppress (#76606 decision B) deletes messages on purpose, and the mitigation this gate is a precondition for deletes one more. DEFINITION, per request, CC's raw array R against the forwarded array F. R-side: every content unit of a non-assistant message is forwarded byte-identically, or belongs to a DECLARED suppression (stats.suppressions, the extension's own report) whose bytes are reconstructible from F — as a forwarded unit, as the "\n\n" join of one message's reminder blocks (78940a0's merged standalone), or as the CROSS-MESSAGE join of two adjacent forwarded messages — or is a declared strip. F-side: every unit is present in R, or was sent by CC in an EARLIER request of the same conversation — which is what "the pin forwards the FIRST-SEEN bytes" means, stated as a checkable property instead of trusted — or is a declared tool_addition injection. Population is non-assistant messages, definitionally rather than conveniently: every mechanism that can delete or re-serve content here is confined to it (classifyPinned skips assistant entries before suppressing; pinnedForwardForm passes anything but a user entry through untouched). Measured over 936 requests of four live captures, the only blocks the pipeline does not conserve are assistant tool_use (rewritten in place by tool-input-normalize) and assistant thinking (sanitized) — a separately-gated class with no telemetry to key an exemption on, so including it would fire on two declared behaviours. The count of skipped blocks is reported rather than hidden, so the boundary is visible to a reader of the row. THE EXEMPTION REGISTRY IS NOT A GUESS — the gate populated it by going red. Its first sweep over the live corpus was clean on 29 of 30 captures and reported 645 violations on s-633915a8, all of kind `lost`, all at message 0. Stage-by-stage replay of request 822 named the cause: RAW 6 units, after fresh-session-sort 3, the three removed being a /compact `<local-command-caveat>`, its `<command-name>` and its `<local-command-stdout>` — the harness quoting its own slash command back, which that extension deletes on purpose (fresh-session-sort.mjs, "Strip /clear artifacts from first user message"). Declared behaviour, not lost conversation, so it is exempt via that extension's own exported predicate rather than a restatement of it. Ruling the instrument out before filing the defect is what kept this from being reported as a content-loss bug in fresh-session-sort. Red-first, seven mutations each biting its own test: R-side loss detection neutralised -> the two loss bites; suppression reconstruction neutralised -> both suppressed-without-copy bites; F-side invention detection neutralised -> both invention bites; the first-seen registry made global instead of per-conversation -> the co-tenant bite; join reconstruction dropped, cross- join dropped, cross-join adjacency dropped -> their three tests respectively. THE REAL RED, on the committed flap bytes rather than a synthetic shape: a throwaway variant applying naive merged-standalone suppression over fixture flap-s-0d6f38ba-86.json (suppress the three migration arrivals, re-serve nothing) reports n=104 suppressed-without-copy: in[91] (system): 1 of 1 unit(s) reconstructible from neither a forwarded block nor a forwarded join n=108 (same) while msg86 and msg94 pass — their copies really are on the wire. msg91 is the CROSS-MESSAGE join, and its second constituent is msg90, whose bytes naive suppression drops from the conversation entirely. That is the evidentiary answer to the parked design question, and the shipped pipeline is green on the same four requests. Rides gate-live as a row field with the same rank as safety, and fails the sweep on its own. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 95ca0cb)
…Of; amend the hand-rolled-identity rule
The row-4 canonical rule was hand-derived from two occurrences in one
capture: it reproduced one byte-exactly and failed the other. That split is
invisible at n=1 and is the difference between a mitigation that absorbs a
bust and one that moves it, so the design could not proceed on it. The
hand-derivation was the prototype; this is the mechanism.
tools/reminder-migration-census.mjs byte-tests the canonical rule across a
capture corpus and reports four verdicts, deliberately separated so none can
inflate another:
EXACT reconstruction is byte-identical to CC's own later message
EXTENDED CC's later form carries NEW text — a different class entirely
DROPPED blocks vanished with no counterpart — rule never exercised
MISMATCH a real hole; any occurrence blocks shipping
Result over the corpus (22 captures, 74 conversations, 3431 pairs):
24 EXACT (85.7%), 3 EXTENDED, 1 DROPPED, 0 MISMATCH. The rule holds on every
occurrence it applies to; the EXTENDED cases are new-information arrivals and
stay booked as their own class.
Getting there took three corrections to this tool, all one family — it
hand-rolled conversation identity instead of importing it:
1. compared before[i] to after[i] by INDEX; one inserted message shifts
every later index
2. paired ADJACENT capture lines; interleaved tenants put two requests of
one conversation several lines apart (replay.mjs already documents this)
3. scored DROPPED blocks as rule failures, manufacturing a blocker
The first two together reported 475 failures / 99.3%, every row reading
actual=0ch — the tell that no counterpart was found at all. Corrected: 0.
So replay.mjs now EXPORTS conversationOf with the rule stated at its
definition: any tool comparing two requests must group by conversation, never
by capture adjacency and never by index. Exporting beats restating — a second
tool re-deriving the identity is how the two drift.
docs/dev-loop.md's "Never hand-roll identity in a probe" is AMENDED rather
than joined by a second section: the new instance is added (a tool, not a
throwaway probe, which is what let it look authoritative), and two corollaries
it forced are stated — extend an existing tool before writing a new one,
because reuse inherits hard-won correctness while a new file re-earns it from
zero; and group by conversation, exporting an identity rather than restating
it when one is missing.
Verified: --selftest green (canonical join, unwrapped passthrough, the three
classify verdicts incl. EXTENDED-is-not-EXACT, host detection); full-corpus
run reproduces the numbers above; replay.mjs still imports cleanly with the
new export.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq
(cherry picked from commit 7b2a5ef)
…d verdict
On 2026-07-31 a single bust took a six-step hand investigation — statusline,
worktime ledger, CC transcript, proxy journal, capture pair, body diff —
before replay --census could even be pointed at it. Six of the ten steps were
manual, and the two most valuable findings came out of steps nobody repeats
under time pressure. One of them was an entirely uncovered bust class, found
only because a diff happened to be read: nothing in the stack had a "none of
the above" branch, so a class with no matrix row was invisible by
construction.
bust-triage chains the existing tools rather than reimplementing them
(dev-loop.md, "Never hand-roll identity in a probe"): classification is
replay.mjs's censusPair, the container byte-test is
reminder-migration-census's canonical/classify, and only the
ledger/transcript reconciliation and the matrix lookup are new here.
THREE answers, never two:
MITIGATED known class, matrix row not open
KNOWN-OPEN known class, row N, still open — prints that row's status
UNCLASSIFIED no row matches; an unrecognised class is the payload
UNVERIFIABLE a step could not run; never folded into a pass
It also reconciles the ledger against the transcript, because those two
disagreed live today — the display upgraded a raced cause while the record
kept "other", and the divergence was invisible until something compared them.
Three pair-selection defects were found and fixed by running it against a
bust whose answer was already known by hand:
1. +30s slack selected a request 35s AFTER the bust — an append-only pair
that reported UNCLASSIFIED, a phantom new class. The busting request
always PRECEDES the ledger stamp; the hook runs after the response.
2. no sidecar guard: one session id spans the main thread, subagents and
1-message bootstrap calls, so the newest request before the stamp was
often a sidecar. A 44k rewrite classified as "identical" on n=1->n=1.
3. (in the census it imports) conversation grouping rather than adjacency.
Each produced a confident wrong answer that looked like a finding.
Verified: --selftest green (classToRow invents no row for unknown classes,
retraction and hit-cause handling on a synthetic ledger, matrixRow reads row
4 as OPEN). End-to-end against the known bust it reproduces the hand-derived
result exactly — transcript messages_changed/105006, pair 11:40:45->11:41:05
n=130->124, census replace/edit, row-4 container migration at host 97 (EXACT),
verdict KNOWN-OPEN. A bust whose capture has rotated away reports
UNVERIFIABLE rather than guessing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq
(cherry picked from commit a774176)
Pre-compact work that ran today but was never committed: hostId (tool_use_id of the leading tool_result) locates where a host landed in the after-request; a standalone candidate must sit AFTER its host — content matching alone picked an identical system message hundreds of slots away (offsets like -839). EXACT findings now carry the offset and the summary prints a placement tally, since emitting the right bytes at the wrong index diverges the prefix just the same. Verified by execution: today's runs over the 77fe2779 capture used this code (placement: +1, single placement). Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq (cherry picked from commit 3afce21)
The census byte-gate slurped each capture with readFileSync and swallowed the failure, so on the live corpus the four LARGEST captures — 6.2 GB of 7.9 GB — fell out of every verdict it ever produced while it reported "25 capture(s)" and exited 0. Same RangeError replay.mjs was fixed for on 2026-07-28, re-committed in a newer tool, plus the three-answer violation: an absence reported as a pass. The read now shares read-lines.mjs with the gate, grouping keeps only each conversation's PREVIOUS request rather than the whole file, and a run that could not read something names it in the header, in a COULD NOT READ block, in --json, and in the verdict block — and exits 1. Measured over ~/.claude/cache-fix-captures/*.jsonl: before 25 capture(s), 82 conversations, 3646 pairs (4 skipped SILENTLY) after read 39/39, 0 UNREADABLE, 165 conversations, 10290 pairs tallies on the previously-readable files unchanged: 17 EXACT / 10 EXTENDED / 1 DROPPED / 0 MISMATCH (baseline re-run on the same 35 files agrees at 3662 pairs vs 3661 — live captures grew between the two runs) 2.4 GB capture censused in 6 s under --max-old-space-size=512 FINDING from the newly-readable 79%: placement is no longer single. 55 EXACT at host+1 and 3 at host+4, so the tool now prints MORE THAN ONE PLACEMENT where it used to print "single placement; safe to emit" — a normalization emitting at host+1 would diverge the prefix on those 3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit a77c930)
EXTENDED was one label for two phenomena, and only one of them is new information. The remainder beyond the canonical reconstruction is either a standalone role:"system" message the PREDECESSOR already sent (CC merged an existing message into the migrated one — nothing new crossed the wire) or content no earlier request carried. That distinction decides a mitigation, had been hand-derived once (extended-absorb-report §b1) and lived nowhere, so the next session would have derived it again. The header's "NOT absorbable by any normalization — new information, not re-serialization" is corrected: refuted for the merged sub-class, and the correction is measured rather than argued. Measured over ~/.claude/cache-fix-captures/*.jsonl (39/39 read): 21 EXTENDED — 21 MERGED-STANDALONE, 0 NEW-TEXT the report's 9 known occurrences each reproduce as MERGED-STANDALONE extendedSub rides --json, so bust-triage can key on it Sub-verdicts come from subclassifyExtended, checked against the BEFORE request's standalones only: matching the after request's own would make every merge trivially true, since the classified message is one of them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit a301ef1)
Row 22's refutation came from a throwaway drop-scan probe with hand-rolled per-message hashes — the tell that a check was missing. It is a check now: every same-conversation pair whose message count DECREASED is classified PURE-TAIL-PRUNE / INTERIOR-DIVERGENT / UNANCHORED, with the re-billed suffix length on every row, using firstDivergence and isHumanTurn imported from replay.mjs rather than restated. gate-live runs the census as a second child per capture under the same heap cap, so the migration byte-test and the prune summary land in the daily sweep and in cache-fix-gate-status.json. Coverage failures bite (a capture the byte-gate could not read makes the row not clean — that is item 1's done-criterion); findings (MISMATCH, interior prunes) are carried, never failed, since they are facts about CC's traffic and a check that fires on non-defects trains its reader to ignore red. DEVIATION from the backlog entry's verifier, with its basis. The entry predicts 12 events, 10 pure / 2 interior on s-77fe2779. Events reproduce exactly (12) and 11:41:05 reproduces as INTERIOR-DIVERGENT (breaks at 97, anchor 123, re-bills 27 of 124). 11:31:58 does NOT: read at the bytes it is the same phenomenon as the ten pure ones — CC pruned a [SUGGESTION MODE: …] scaffolding block and the user's real turn landed at the same index — and differs only in the live turn having produced 3 messages instead of 1-2. Splitting it off requires a "within N of the tail" threshold that no definition produces, so the boundary here is the ANCHOR (isHumanTurn), the same relation row 4's verdict rests on. Result: 11 pure / 1 interior. Corpus-wide, 39/39 captures: 226 drop events, 181 pure, 45 interior, 0 unanchored. Two interior events re-bill nearly everything — 2026-07-31 12:42:11 (n=688->675, breaks at 4, re-bills 671) and 11:40:24 (n=83->81, breaks at 4, re-bills 77); the first lives in a capture that was unreadable until the previous commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit 404d5fc)
bust-triage read only k:"hit", so on 2026-07-31 the statusline showed `❄ 55k compact (8m)` (ledger k:"cost", t=1785505434) while --list showed nothing newer than 90 minutes earlier and the default run silently triaged an older, unrelated event. An event the operator can SEE must never be missing from the tool that explains events. coldEvents() now reads the whole ❄-visible population and splits it: "bust" (k:"hit", triageable) vs "controlled" (k:"cost" plus legacy k:"resume" — a cost the operator or the auto-compact ceiling caused). busts() keeps its old meaning, so nothing downstream shifts. --list labels controlled events CONTROLLED(<cause>), and when the newest cold event is controlled the default run states "cannot triage: controlled cause" and names the bust it fell back to, in text and in --json. The controlled set comes from claude-worktime itself, not from this tool: the ❄ token advances on cold_hit and cold_cost, and its `--cold --all` filter lists hit, cost and legacy resume. That is why resume is included where the backlog entry named only cost — the done-criterion is that a ❄-visible event can never be absent, and 3 resume records are in the live ledger. Verified against the live ledger: --list carries "2026-07-31 13:43:54 55k CONTROLLED(compact) 77fe2779", and a no-args run over a copy truncated to that instant prints the NOTE and falls back to the 12:25:23 bust — the same substitution that used to happen in silence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit 6efce90)
…itization scrubText tokenized whole texts, so scrub(a + "\n\n" + b) != scrub(a) + "\n\n" + scrub(b) — measured, not inferred (extended-absorb-report §c5). The prefix and join relations that DEFINE EXTENDED and the merged-standalone shape died at scrub time, so a fixture pinned for that class could not reproduce the class it was pinned for and extended-absorb.test.mjs had to hand-build synthetic tokens instead. The scrub now splits on "\n\n" — the domain's join, the same literal the census's canonical()/classify() and insertion-normalization's duplicate suppression already hardcode — tokenizes each segment, and rejoins. Wrap handling runs first and unchanged, so the fixed-constant lesson holds at paragraph granularity. Inputs outside the join contract degrade to the old whole-text behaviour: no crash, no leak, relation simply not promised. The accepted privacy delta is metadata only — paragraph count, per-paragraph lengths, cross-text sharing of identical paragraphs, never content bytes — accepted by operator ruling for this local, controlled deployment, with the audience caveat carried in the scrubber's comment for anyone harvesting third-party traffic. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit bffcb05)
a301ef1's test file said a harvested fixture cannot reproduce the merged- standalone class, citing the then-PARKED §c5 (scrub tokenized each text independently, so the prefix/join relation did not survive). bffcb05 landed a "\n\n"-homomorphic scrub while these items were in flight, so the premise is dead and the comment would have taught the next reader the opposite of what the code does. Verified against the shipped scrubMessage rather than from the commit message: prefix and join relations both survive, classify() returns EXTENDED on the scrubbed bytes and subclassifyExtended returns MERGED-STANDALONE. The fixtures stay synthetic — a unit test wants a minimal pair it controls — but that is now a preference, not a constraint. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit 496fbf0)
…ow (verification slice of fork a1170a7) Only the tools/ half of fork a1170a7 travels in this slice: replay's stability and conservation checks learn the join-move action so a re-served move reads as designed behaviour, not as a violation. The extension half and its tests ride in the insertion slice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit a1170a7, tools/ paths only)
…, diffed
Graduates the throwaway A/B script of the unit-2b build (closing report
2026-07-30, "Corpus A/B — nothing else moved") because the reserved-entry
identity build needed it a second time: a probe used twice graduates or dies
(dev-loop). It answers a question replay.mjs is single-tree by construction and
cannot ask — does CHANGING the code change any decision it takes on the
committed corpus — by holding two extension modules resident at once and
diffing the verdict line (action, reset reason, pinned, suppressed, moved,
dropped, forwarded length) per request.
Two modes, two different questions. Independent chains asks whether
steady-state behaviour moved. `--seed-from-a` feeds tree B, at every request,
the canonical tree A wrote for the preceding one — the OLD-CANON COMPATIBILITY
probe, i.e. whether a restart is transparent for conversations already in
flight, which is threat-matrix row 3's question and previously answerable only
by argument.
Three things carried over from the lessons that produced it. It exits 2 with
COULD NOT VERIFY when no fixture yields a replayable request, because the first
version of the unit-2b probe printed "IDENTICAL" over two empty dumps after
crashing on both trees — demonstrated red here against an empty fixture
directory, not asserted. It reads all THREE committed fixture shapes
(`{requests}`, `{header, records}`, `.jsonl` capture records) and names every
file it skipped, because the first draft of the reader silently saw 2 of the 6
message-array corpora and would have reported a 9-line "IDENTICAL" as a
44-line one. And it groups by the extension's OWN `resolveInsertionSessionKey`
rather than chaining one canonical per file — the pinned fixture alone carries
six conversations, and comparing across them is the hand-rolled-identity error
this repo has paid for four times.
Trees are given as git refs (checked out detached into a scratch worktree and
removed afterwards) or as directories; the shared working tree is never used
as a scratch checkout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit da8b837)
…ery, and the absence scan — match fork 687cbc5/eb4f844 The slice ships the harvester, so it ships the FIXED harvester: scrubBlock recurses into source (the payload one level below where the old scrubber looked — the measured five-PNG leak class) and fails closed on any long string there. Capture discovery in the two real-pair tests recovers the file by hashing candidates against the fixture's own token instead of hardcoding a capture id (a capture UUID plus a home path is a live identifier in a public tree). tools/absence-scan.mjs + its test make sanitization CHECKED rather than claimed, per the cnighswonger#272 fixture-strategy thread; one allowlist entry added with provenance (upstream's own org_id example in docs/directives/proxy-cache-warmer-v3.7.0.md). The grafted nesting tests pin the source.data class red-first at unit level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y33RMV399iYMXFEbAfQCk
… import # Conflicts: # test/insertion-suppression.test.mjs
…he RESET path too The suppression built for #76606 (decision B) was silently disabled by any reset. `resetKeepingPins` restores the pins and returns BEFORE the migrated-duplicate pass, so CC's standalone copy of an already-pinned reminder went out on the wire beside the restored inline form — the reminder forwarded twice, mid-array, moving the cache's longest-identical-prefix boundary to just before it. Measured live 2026-07-31 (session 77fe2779, request 11:41:05.778Z): the telemetry for that exact request reads action=reset resetReason=not-subsequence pinned=2 suppressed=0 Pins restored, suppression skipped. Outcome: edit@98 of 123, transcript cache_miss_reason messages_changed / cache_missed_input_tokens 105006, ~104 kB re-billed. Scale, from this file's own measurement: 125 resets across 350 requests, roughly one request in three. A mitigation that switches off on a third of requests, without saying so, reads as shipped and behaves as absent. The fix reuses what the reset path already holds — the pins it just restored, and priorCanonical — rather than adding a mechanism: a standalone duplicate is suppressible exactly when the inline form it duplicates is being restored. Suppressed entries are removed from the forwarded array and excluded from the canonical, preserving the success path's invariant that the canonical describes the wire we just forwarded. NOT claimed: what made the survivors non-subsequence on the live request. `not-subsequence` requires matched entries to invert in order, which a plain pruning does not produce, so the "the suggestion-mode pruning caused it" story is unverified and is stated as such in the test header. The fix does not depend on the trigger. Verified — and two false starts are recorded because both nearly shipped a claim: (1) the first regression scenario never reached the reset path at all (it classified as "normalized"), so it proved nothing until rebuilt around an explicit reorder that yields resetReason="not-subsequence"; (2) the first instrument proof reverted the code into a syntax error, and a red from a file that will not load proves nothing — redone by neutralizing only the suppression with the module still loading. With it neutralized the two regression tests go red and the three guard tests stay green; restored, 5/5 here and 73/73 across all four insertion suites. Guards against the opposite failure: a reset with nothing to suppress is unchanged, a standalone matching no pinned block is forwarded untouched, and a tail-position duplicate is never suppressed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq (cherry picked from commit 059aae3)
…the EXTENDED-absorb design that measurement killed (insertion slice of fork 5c4d70a) The reset path suppressed migrated duplicates without declaring WHICH incoming indices it suppressed. replay.mjs keys two gates on that declaration — safetyViolation() filters the indices out of the input side before comparing lengths, conservationViolations() accepts a missing unit only when it is part of a declared suppression — so both reported a designed behaviour as corruption. Replaying capture s-77fe2779 (conversation e7394e05, request 11:41:05.778Z) under the serving gate set: 1 safety violation (length 124 -> 123) + 1 conservation violation (lost in[98]) before, 0 and 0 after, forwarded bytes unchanged. 059aae3 added the suppression at 13:35Z, after the day's 07:52Z sweep, so no gate run had exercised it yet and the next daily sweep would have gone red. It also restores the per-suppression event lines the "rule out ourselves" attribution sweep reads, on the ~1-in-3 requests that reset. The dispatched EXTENDED-class absorb is NOT built, and the directive, BACKLOG entry and matrix row 4 now carry why. Its premise was that the EXTENDED remainder is new harness text; measured, it is a standalone system message the predecessor already carried, swallowed into the migrated reminder (9 of 9 occurrences corpus-wide, 0 new text). Its action was to re-emit that remainder at a frozen tail index; measured on the real pipeline, the first forwarded divergence stays at 100, while restoring the swallowed message at ITS index moves it to 123 of 124. That un-merge is unit 2 of flap-move-mitigation-and-fidelity-gate.md, already built on wt/fidelity/opus and blocked on the identity decision — so the item is a duplicate that closes by merging there, not a second mechanism. Evidence: docs/code-reviews/extended-absorb-report.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit 5c4d70a, code + test paths only; the BACKLOG / threat-matrix / directive halves are fork-internal and stay behind)
…and moves surviving resets) reconciled with the reset-path duplicate suppression Cherry-picks of aef760b and dc8c475 from branch wt/fidelity/opus (author Gunther Schulz, Co-Authored-By Claude Opus 4.8 on both), squashed into one integration commit because the second does not stand alone on current main. Their content, verbatim except for the reconciliation below: aef760b a recognized reminder move serves its first-seen form — no reset, no lost bytes. Cross-message join recognition (findJoinMoves), in-place substitution of the absorbed entry into the merged message's slot, `moved`/`reserves`/`suppressions{kind:"join-move"}` telemetry, and replay.mjs's wireRemovedIndices so a substitution is not mistaken for a removal. dc8c475 a reset keeps recognized moves — same argument, same riders as the pins it already keeps. findJoinMoves runs inside resetKeepingPins after the pin substitutions, on the same array. RECONCILIATION with 5c4d70a (reset-path duplicate suppression, which landed on main after dc8c475 was written and touched the same return statement). Both suppression kinds now flow through the ONE `suppressions` array the gates read, exactly as the success path already does: - the merged slot is declared `kind: "join-move"` and is NOT added to the removal set — the slot survives, carrying the re-served first-seen bytes; - a plain duplicate is declared without a kind and IS removed from the forwarded array; - `suppressed` counts the declaration array, matching the success path; - the reset's canonical is built from the KEPT entries (duplicates excluded, the success path's invariant) with a moved slot filing the ABSORBED entry rather than a fresh identity built from the merge (dc8c475's invariant); - the `messages` guard is `applied > 0 || moves.length > 0 || suppressedR.size > 0` — the union of both features' guards. Neither feature's behaviour is narrowed: the move branch is consulted before the duplicate predicate, so a merged slot can never be routed down the removal path, and a duplicate is untouched by the move machinery. Baseline after reconciliation, three targeted files (join-move, normalization, merge-suppression): 78 tests, 75 pass, 0 fail, 3 todo — the three TODOs are dc8c475's, still todo, as expected before the identity fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit a1170a7; its tools/replay.mjs half arrived with the pr/verification-tools merge)
…pace, so one more copy of its text cannot re-bind it Unit 2b was built and bite-proven and did NOT close the regression it was written for, because the mechanism it blamed was not the one firing. Measured on capture s-dc3f8071 (fixture reset-move-s-dc3f8071-196-197.json): a recognized move keeps an entry alive in OUR canonical while CC has stopped sending it, and that entry's key is (content-hash, role, occurrence-ordinal- within-the-request). An ordinal is a claim about CC's array — an array the entry is not in. The claim went false at n=197, when CC sent an eighth copy of a recurring tail reminder: the copy took o=7, the entry bound to it 13 slots away, and two things broke at once — the entry left `droppedNow` so no move recognition could fire, and the inverted pair tripped `not-subsequence`. Same shape, same merged-content hash, again at n=399->400. So a re-served entry's identity is now its stored first-seen bytes plus the canonical slot where we last forwarded them, marked `rs: true` at the mint, and it does not participate in (hash, role, ordinal) wire matching AT ALL — not looked up, not counted as dropped, not read by the canonical-order check. A fresh copy of the same text takes the next free ordinal, matches nothing, and classifies as an ordinary new entry on the existing append/splice path. Every non-reserved entry keeps absolute matching byte-for-byte: the general ordinal instability of duplicate copies under middle-copy drops is a pre-existing class and deliberately out of scope. What replaces the match is a per-request disposition over the entry's neighbourhood, resolved exactly as findJoinMoves' condition (d) resolves it, one of three in this order: RE-FIRE (the merged form is on the wire again -> re-serve the stored bytes into that slot, declare the join-move, stay reserved), RECLAIM (CC flipped back to the original form -> clear `rs`, bind the entry to that wire index as an ordinary matched entry and rewrite its stored key from that message's incoming identity), LAPSE (neighbourhood resolvable, neither form present -> not carried into the rebuilt canonical). Bounds unresolvable or crossed: nothing happens at all — no substitution and no state change, raw forward. The lapse rule is the mitigation for the one new risk the design introduces, re-serving stored bytes into a context CC has pruned away, and it fails closed whenever its preconditions are not byte-established on the current wire. Lifecycle is self-limiting by construction: an `rs` entry persists exactly as long as its re-serve fires. findJoinMoves gains condition (f) — merged wire message role "system" and absorbed entry stored role "system" — closing the latent role gap the unit-2b report surfaced (§c5). The same constraint binds both disposition probes. Red-first, ten mutations each biting the bite that names its condition: match exclusion, the mint's `rs`, re-fire, reclaim, lapse, fail-closed bounds, (f) at the mint on both sides, (f) on the probes, and the canonical-order exclusion. The reclaim bite's first draft survived its mutation — a lapse followed by a fresh entry with the same bytes was observationally identical in that shape — and now asserts the property that actually separates them: the reclaimed message is a MATCHED entry, so it is not an insertion and the request is a plain tail append. Two of unit 2b's three TODO tests flip to passing unedited, including the five-gate one: the fixture's stability violation at n=197 is gone. The third does not, and its expectation is not edited — its control asserts `action === "reset"` at n=197, and with the re-bind removed there is no inversion, so n=197 classifies `normalized`. The reset was the symptom the unit-2b report named as such; the criterion that test carries (stability quiet) is met by the sibling that asserts it directly. Threat-matrix row 3, restart declaration: no key-scheme change. (h, r, o) storage is unchanged and `rs` is a new OPTIONAL field, so canon files written by the old code contain no `rs` entries and take identical decisions under the new code — measured, not assumed, by tools/verdict-ab.mjs --seed-from-a. Production has never run unit 2, so no deployed canon file can carry a re-serve. The restart is cache-transparent for every existing conversation; the new machinery activates only at the first post-restart move recognition. Directive: docs/directives/reserved-entry-identity-directive.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit fad6f6b)
…t a scan over `matched` Self-caught while re-reading the previous commit: the reset path's canonical rebuild resolved "was this wire index reclaimed?" with a `matched.find` inside a `.map` over every kept entry — O(entries x matched) on a path that runs on roughly one request in three (this file's own measurement: 125 resets across 350 requests), against requests that reach 2000+ messages on the large captures. The disposition pass already knows the answer, so it records it: wire index -> ci, one map lookup at the rebuild. Pure refactor of one expression, no decision changes — proven by `tools/verdict-ab.mjs` against the parent commit rather than argued, and the ten mutations of the parent all still bite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit 0cc05c7)
…e definition, all three unit-2b todos retired green The control asserted action==='reset', the exact signature the reserved-entry identity build removes; verifier 1's definition (n=197 normalized, re-fire continues, bytes at the merged slot hold) is what the test now pins. Its red against the pre-build tree is on record in the A/B (actual normalized vs expected reset). TODOs 16/17 flipped unedited and lose their todo markers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0171Mpvi9GsSnJjBNnbfxU2u (cherry picked from commit 8e3c265)
…the mint guaranteed (insertion slice of fork 9983a1b) The two conditions read as defensive padding for an impossible case. They are not: the mint happened in a PREVIOUS process and the entry arrived through a canon file on disk, so a deserialization boundary sits between the guarantee and this check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit 9983a1b, code path only; the report and directive are fork-internal)
…ipt-shape fixture Their identifiers, committed upstream, public in the upstream tree before this scan existed — a pre-existing-third-party file the scan must name, not go red on forever. Provenance beside the entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y33RMV399iYMXFEbAfQCk
Gunther-Schulz
added a commit
to Gunther-Schulz/claude-code-cache-fix
that referenced
this pull request
Aug 1, 2026
…ified, pushed; draft cnighswonger#295 open Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y33RMV399iYMXFEbAfQCk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
CC sometimes merges a
<system-reminder>message and its immediatestandalone neighbour into a single message mid-history, then sometimes
un-merges them again a few requests later. Every one of those flips rewrites
history the model has already seen, which busts the prefix cache and re-bills
the whole conversation.
#272 gave the proxy the ability to recognise a message across
re-serializations and to pin the first-seen bytes. This PR closes the two
cases that pinning alone could not:
1. The join-move un-merge. When we recognise that a message CC used to send
standalone has been absorbed into its neighbour, we serve the first-seen
(un-merged) form upstream instead of the newly merged bytes. The join grammar
is a single
"\n\n"separator — the same literal the duplicate-suppressionpath already keys on — and the probe is byte-exact: the merged wire message's
text must equal
pinnedReminderText(predecessor) + "\n\n" + absorbed.text, bothmessages must be
role: "system", and the absorbed entry's neighbourhoodbounds must resolve on the current wire. Anything else fails closed: no
substitution, raw forward, today's behaviour.
2. The identity fix — a re-served entry leaves the wire-identity space.
This is the part that took a rebuild rather than a patch. Pinned entries are
keyed by
(content-hash, role, occurrence-ordinal-within-the-request). Arecognised move keeps the absorbed entry alive in our canonical while CC has
stopped sending it — so its ordinal is a claim about an array it is not in.
The moment a later request carries one more copy of the same recurring text
(measured: a fresh tail reminder taking
o=7), the stale entry binds to thatunrelated copy at an inverted position, which both removes it from the dropped
set — so no move recognition can fire — and trips the subsequence check. The
merged message then goes out raw and our bytes flip at an index where CC's
were identical.
The fix marks such an entry
rs: trueand takes it out of(h, r, o)matchingentirely. Its identity becomes its stored first-seen bytes plus the canonical
slot where we last forwarded them. Each request it gets exactly one of three
dispositions, checked in order: re-fire (the merged form is present again →
re-serve), reclaim (CC flipped back to the original form → clear the mark
and rebind as an ordinary matched entry), lapse (neither form present → the
entry is dropped, never re-served into a region CC no longer carries). Entries
that were never re-served keep absolute
(h, r, o)matching byte-for-byte.Measured — A/B over the live corpus
Two detached worktrees differing only by this diff, replayed over 8.5 GB of
real capture (36 captures, ~10 000 requests) under the serving gate set:
Cross-request byte-stability violations go 10 → 2, and the two survivors
are attributed by the gate's own attribution line to a different extension
(
deferred-tool-rewrite) and are byte-for-byte the same two pairs in bothtrees. Zero insertion-normalization stability violations remain in the whole
corpus. Safety, conservation, sequence and canonical order read 0 on every
capture in both trees.
Worth stating plainly: the same ordinal collision was firing on four
captures, not the one it was found on. That only became visible because the
measurement was corpus-wide rather than fixture-wide.
Two honesty notes about the sweep as an A/B. The capture count differs (33 vs
36) — three tiny captures (2, 13 and 1 requests) were present only for the
second run, all clean, none in the failing set. And several captures are live
and still growing, so two sequential 8 GB sweeps are confounded in principle —
but the per-capture request counts are identical for every capture in the table,
so for the captures the comparison is about, both trees replayed the same input.
Old-canon compatibility was measured, not argued:
tools/verdict-ab.mjs --seed-from-areplays decisions over canon files written by the pre-changecode and is identical across 44 verdict lines / 6 corpora.
rsis a newoptional field; canon files from the old code contain none, and under the new
code they take identical decisions. A restart shipping this is
cache-transparent for every existing conversation.
Non-Functional Requirements
proxy/extensions/insertion-normalization.mjs(no new production file, nonew abstraction, no new env var — it extends
classifyPinned,resetKeepingPinsandfindJoinMovesin place). The originating directivebudgeted 120–200 LOC; the overrun is the reconciliation with feat(insertion-normalization): pin volatile reminder blocks so mid-history rewrites stop busting the cache #272's
reset-path duplicate suppression, which had to unify two declaration paths
rather than add a second one. Tests: ~1,170 lines across three files, plus a
20.6k-line harvested fixture — see the fixture caveat below.
conservation and stability gates are the enforcement. The new risk this
design introduces is re-serving stored bytes into a context CC has pruned or
compacted away. The lapse disposition is the mitigation and it fails closed
— no re-serve — whenever its preconditions are not byte-established on the
current wire. No new persisted state shape beyond
rs, one optional booleanon an existing entry.
JOIN_SEPARATOR);the merged-form probe is the same literal the duplicate suppression already
uses, seen from the other side.
neighbourhood) per request; reserved entries measured at 1–2 per conversation
in every observed instance.
forwarded on the wire.
Open, and inherited from #272
reminder-only-edit case (blocker 1) and the canon-file permissions (blocker 3)
are properties of the base this branch sits on. Whatever lands there lands
here.
test/proxy-read-dedupe.test.mjs:505is red on this branch, exactly as itis on feat(insertion-normalization): pin volatile reminder blocks so mid-history rewrites stop busting the cache #272 — the extension-order adjacency assertion. Reproduced at feat(insertion-normalization): pin volatile reminder blocks so mid-history rewrites stop busting the cache #272's
head before any commit here: 41/42 on that file alone. Left for the
deliberate fix feat(insertion-normalization): pin volatile reminder blocks so mid-history rewrites stop busting the cache #272's review asked for (move the order, or update the
assertion) rather than patched in a stacked PR.
test/fixtures/harvested/reset-move-s-97097e027ac0-196-197.json(the resetleg) and, via the feat(tools): the verification stack — replay gate, census, harvest, scheduled sweep #276 merge,
flap-s-0dc8ac87c43d-86.json(the oscillationleg) — both fully rebuilt through the hardened scrubber (token-named, nested
payloads tokenized, timestamps rebased), per the path agreed on feat(insertion-normalization): pin volatile reminder blocks so mid-history rewrites stop busting the cache #272's
thread. The feat(tools): the verification stack — replay gate, census, harvest, scheduled sweep #276 merge also carries
tools/absence-scan.mjs+ its test, sothe sanitization claim is checked in-tree, not asserted: the branch's own
scan over its whole delta reads clean.
🤖 Generated with Claude Code